Running OpenMp Jobs

OpenMP (Open Multiprocessing) is an add-on in compiler. It targets shared memory systems i.e where processor shared the main memory and is based on thread approach.

It launches a single process which in turn can create n number of threads as desired. Depending on a particular task it can launch desired number of threads as directed by user.

OpenMP is a set of compiler hints and function calls to enable you to run sections of your code in parallel on a shared memory parallel computer. In OpenMp all threads share memory and data.

In the below example, the c program starts up 4 threads in parallel.

Example

  • C Script 'program.c'

  • Batch script 'script.sh'

#include <stdio.h>
#include <omp.h>

int main(int argc, char *argv[])
{
  #pragma omp parallel
  {
    int NCPU,tid,NPR,NTHR;

    /* get the total number of CPUs/cores available for OpenMP */
    NCPU = omp_get_num_procs();

    /* get the current thread ID in the parallel region */
    tid = omp_get_thread_num();

    /* get the total number of threads available in this parallel region */
    NPR = omp_get_num_threads();

    /* get the total number of threads requested */
    NTHR = omp_get_max_threads();

    /* only execute this on the master thread! */
    if (tid == 0) {
        printf("%i : NCPUt= %in",tid,NCPU);
        printf("%i : NTHRt= %in",tid,NTHR);
        printf("%i : NPRt= %in",tid,NPR);
    }
    printf("%i : hello user! I am thread %i out of %in",tid,tid,NPR);
  }
  return(0);
}
#!/bin/bash
#SBATCH --job-name OmpJob		## Job name
#SBATCH --output Omp.out		## Output file
#SBATCH --time 00:30:00			## Amount of time needed DD-HH:MM:SS
#SBATCH --cpus-per-task 4 	## Number of threads
#SBATCH --mem-per-cpu 100M 	## Memory per cpu

module load openmpi

gcc -fopenmp program.c -o program

export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK

srun --mpi=pmi2 ./program

Output

3 : hello user! I am thread 3 out of 4
2 : hello user! I am thread 2 out of 4
1 : hello user! I am thread 1 out of 4
0 : NCPU        = 4
0 : NTHR        = 4
0 : NPR         = 4
0 : hello user! I am thread 0 out of 4

Explanation

Slurm by default doesn’t know what cores to assign to what process it runs. For threaded applications, you need to make sure that all the cores you request are on the same node.

The OpenMP script is an example that all the cores are on the same node, and lets Slurm know which process gets the cores that you requested for threading.

OMP_NUM_THREADS environment variable is used to specify the default number of threads to use in parallel regions. By adjusting the value of the OMP_NUM_THREADS environment variable, one can adjust the number of execution threads. But in this case, it’s set to the number of threads that the job wants to execute.

stdout Description

3 : hello user! I am thread 3 out of 4

Thread id

2 : hello user! I am thread 2 out of 4

Thread id

1 : hello user! I am thread 1 out of 4

Thread id

0 : NCPU = 4

Total number of available CPUs for OpenMp

0 : NTHR = 4

Total number of threads requested

0 : NPR = 4

Threads available

0 : hello user! I am thread 0 out of 4

Thread id