Conda in Slurm

Using Conda virtual environments in your Slurm script

The example in this section assumes that you have carried out the steps in Installing packages with python pip

After creating your virtual environment, you can use it in your Slurm script because your program depends on the packages contained in the environment. In your Slurm script, there are two lines you’d want to add right after the declaration of Slurm directives, module load conda and conda activate my_env. my_env is the name of the virtual environment which was created earlier.

  1. Log in to Discovery.

  2. Create a file called script.sh and then copy and paste the code below and save afterward.

    #!/bin/bash
    
    #SBATCH --job-name=CamelCase   		## Name of the job
    #SBATCH --output=CamelCase.out  	 ## Output file
    #SBATCH --time=10:00           		## Job Duration
    #SBATCH --ntasks=1             		## Number of tasks (analyses) to run
    #SBATCH --cpus-per-task=1      		## The number of threads the code will use
    #SBATCH --mem-per-cpu=100      		## Real memory(MB) per CPU required by the job.
    
    ## Load the python interpreter
    module load conda
    conda activate my_env
    
    ## Execute the python script
    python program.py

    On line 11, the conda module has been loaded. On line 12, the custom conda environment, my_env has been activated which contains the packages and dependencies, that the project requires to run.

  3. Create a file called program.py and then copy and paste the code below and save afterward.

    import camelcase
    
    c = camelcase.CamelCase()
    
    txt = "hello world"
    
    print(c.hump(txt))
  4. Make the batch script executable

    chmod +x script.sh
  5. Submit the batch script

    sbatch script.sh

    Output If you show the content of the output file CamelCase.out, you should see a result like the one below on your console.

    Hello World

    Thus, the text hello world is converted to a Camel Case format.